Digital cameras, digital camcorders, and other imaging devices typically include image sensors, such as charge-coupled device (CCD) image sensors or complementary metal-oxide semiconductor (CMOS) image sensors. Between these two types of image sensors, CMOS image sensors are being used more frequently than CCD image sensors in a variety of imaging devices, such as digital still and video cameras, mobile phone cameras, surveillance cameras, web cameras, etc. This growth is based at least in part on the ability of CMOS image sensors to easily integrate electronics, such as programmable signal processing circuits, on-chip, which provides low cost, low power consumption, and high speed readout features that can be critical for many applications.
FIG. 1 shows an illustrative diagram of a CMOS image sensor 100. As shown, image sensor 100 includes a 4×4 pixel array 105 having rows of pixels with four pixels in each row and an address generator 110. CMOS image sensor 100 includes column-parallel readout circuits that read all of the pixels in a particular row into a line memory and process the data in parallel. Using CMOS image sensor 100, rows of pixels are sequentially exposed to light and the rows of pixels in pixel array 105 are readout row-by-row from the top row of pixel array 105 to the bottom row.
More particularly, as shown in FIGS. 1 and 2, address generator 110 is a shift register that sequentially scans all of the rows and generates row-reset (RST) and row-select (SEL) signals for a row address decoder 115. The exposure for each row of pixels is controlled by the row-reset and row-select signals sent from the address row decoder (e.g., RST_1 and SEL_1 for the first row of pixels in pixel array 105). Each row of pixels becomes photosensitive upon receiving a row-reset signal, and stops collecting photons and starts reading out data upon receiving a row-select signal. Because there is only one row of readout circuits, the readout timings for different rows cannot overlap. This is further illustrated in FIG. 3, which shows an illustrative representation that provides time on the x-axis and rows on the y-axis. As shown, note that the exposure time 310 (Δte) is fixed for each row and the readout times 320 (Δtr) are linearly shifted from the top row of the pixel array to the bottom row of the pixel array. This approach, where the image is captured contemporaneously for the pixels in a particular row and not contemporaneously between adjacent rows, is commonly referred to as “rolling shutter.”
Rolling shutter and, in particular, the row-wise exposure discrepancy in rolling shutter are considered to be detrimental to image quality. Rolling shutter exposes pixels in different rows to light at different times. This often causes skew, geometric distortions, and other artifacts in an image of a physical object or a scene. The effects of rolling shutter are particularly noticeable in images of moving objects, which is shown in FIG. 4. A vertical line 410 has been included to illustrate the skew or distortion in the image of the moving train caused by rolling shutter.
Various approaches, such as global shutter, attempt to address the limitations of rolling shutter. Imaging devices that implement global shutter expose each pixel in every row of an image sensor to light at the same time to simultaneously capture an image. In global shutter, the readout times and the exposure length for each pixel is the same. However, despite these advances, global shutter and other shutter approaches remain one-dimensional functions. None of these approaches extend beyond one dimension—i.e., the time dimension.
There is therefore a need in the art for approaches that exploit rolling shutter advantageously for computational photography and other applications. There is also a need in the art for approaches that extend the shutter mechanisms to a two-dimensional sampling of the three-dimensional space-time volume of a scene.
Accordingly, it is desirable to provide methods and systems for coded readout of an image that overcome these and other deficiencies of the prior art.